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ABSTRACT 


Air  Force  Research  Laboratory/Human  Effectiveness  Directorate  Radio  Frequency 
Radiation  Branch  (AFRL/RHDR)  and  Naval  Health  Research  Center  (NHRC)  have  long 
used  Finite  Difference  Time  Domain  (FDTD)  software  to  study  radiofrequency  radiation 
(RF)  bioeffects.  FDTD  is  a  direct  time-domain  solver  for  Maxwell’s  equations.  The 
original  Brooks  FDTD  code  was  developed  as  part  of  the  Brooks  Dosimetry  Project  and 
has  been  repeatedly  enhanced  to  meet  empirical  and  theoretical  research  needs.  Many 
feature  upgrades  have  been  added  to  the  original  Fortran  FDTD  code,  and  recently  the 
code  has  been  completely  rewritten  in  C.  Both  versions  of  Brooks  FDTD  are  parallelized 
for  Beowulf  clusters  and  have  run  on  more  than  one  hundred  processors.  In  addition  to 
code,  a  number  of  segmented  anatomical  models  have  been  created  for  the  Brooks 
Dosimetry  Project.  These  models  include  a  human,  rhesus  monkey,  goat,  and  several  rat 
models  (See  Appendix  E).  The  Brooks  FDTD  codes,  used  with  these  anatomical  models, 
allow  researchers  to  perform  useful  RF  bioeffects  modeling  to  aid  ongoing  RHDR  and 
NHRC  research  efforts.  To  ensure  fidelity  of  FDTD  results.  Brooks  FDTD  has  been 
validated  using  several  methods  which  are  reported  here. 
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OVERVIEW 


Background 

The  Finite  Differenee  Time  Domain  (FDTD)  method  is  a  direct  time-domain  numerical 
approach  for  solving  Maxwell’s  equations.  The  FDTD  method  was  first  introduced  by 
Kane  Yee  in  1966  [1],  and  has  become  increasingly  popular  for  electromagnetic 
modeling  and  simulation  (M&S)  as  computational  resources  have  increased. 

The  FDTD  method  is  used  for  many  types  of  applications,  including  military  technology, 
radar  systems,  communication  devices,  computer  circuit  design,  and  studying  the 
biological  effect  of  exposure  to  electromagnetic  (EM)  energy. 

Two  FDTD  codes  have  been  written  at  Brooks;  one  implemented  in  Fortran  [2]  and  one 
in  C  [3].  These  codes  have  been  used  as  part  of  a  number  of  research  projects  studying 
the  bioeffects  of  electromagnetic  (EM)  energy.  Specifically,  the  code  has  been  used  to 
predict  energy  absorption  rates  within  biological  tissues  due  to  incident  fields  in  the  radio 
frequency  region  of  the  EM  spectrum. 

An  EDTD  simulation  is  run  by  executing  a  command  line  directing  the  program  to  use 
basic  text  configuration  files,  command  line  options,  and  a  binary  model  file.  Elpon 
completion  of  a  simulation  the  program  will  output  a  binary  result  file  containing  the 
specific  absorption  rate  (SAR)  at  each  voxel  of  the  anatomical  model.  SAR  is  a  measure 
of  the  maximum  rate  at  which  RE  energy  is  absorbed  by  the  body  when  exposed  to  radio 
frequency  electromagnetic  fields.  The  maximum  E-field  can  also  be  obtained  in  binary 
format  depending  upon  options  selected. 

Due  to  the  size  of  high  resolution  anatomical  models  and  the  large  memory  requirements 
for  the  EDTD  algorithm,  generally  simulations  exceed  the  capacity  of  a  single 
workstation.  Therefore,  Brooks  EDTD  has  been  parallelized  using  Message  Passing 
Interface  (MPI)  to  distribute  the  problem  over  multiple  processors.  The  software  was 
designed  to  run  on  a  Einux  Beowulf  cluster,  which  is  advantageous  because  it  allows  the 
code  to  run  on  commodity  hardware  and  free  and  open-source  software.  When  a 
simulation  is  initiated,  the  model  space  is  divided  along  the  z-axis,  which  the  longest  axis 
for  the  anatomical  models.  Equal  shares  of  the  problem  are  then  allocated  to  each 
processor  for  the  simulation. 


Objectives 

The  Brooks  EDTD  is  used  to  calculate  the  SAR  within  realistic  digital  anatomical 
models.  These  simulations  have  allowed  for  the  study  of  a  much  broader  range  of 
frequencies  than  would  be  possible  by  experimentation  alone.  SAR  predictions  have 
proven  useful  in  experimental  design,  and  in  setting  safety  standards  for  EM  fields. 
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These  standards  are  essential  for  proteeting  the  health  and  safety  of  the  publie  as  well  as 
those  working  in  a  controlled  environment. 

The  usefulness  of  Brooks  FDTD  is  contingent  upon  its  ability  to  produce  results  within 
an  acceptable  range  of  accuracy.  Therefore,  in  addition  to  creating  FDTD  codes,  we 
must  also  validate  FDTD  results  with  as  many  methods  as  possible.  The  purpose  of  this 
report  is  to  document  the  validation  efforts  that  have  been  completed  for  this  software,  as 
well  as  to  provide  a  reference  manual  for  the  proper  use  of  the  code.  Validation  testing  is 
presented  in  the  Results  and  Discussion  section. 


Approach 

Brooks  FDTD  was  initially  developed  to  model  more  complex  geometries  than  possible 
with  other  existing  models.  We  will  compare  both  Brooks  FDTD  implementations 
against  a  Mie  theory  based  code.  Mie  theory  is  a  complete  analytical  solution  for  solving 
Maxwell’s  equations  for  the  scattering  of  a  plane  wave  on  a  sphere  and  is  generally 
considered  perfectly  accurate  since  it  is  a  closed  form  analytical  solution.  If  the  FDTD 
results  approach  the  Mie  values  within  an  acceptable  margin  of  error,  we  may  assume  the 
FDTD  results  are  accurate  within  that  margin  of  error. 
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Introduction 


As  directed  energy  (DE)  technologies  become  more  pervasive  in  DoD  communications 
and  weapon  systems,  it  becomes  critically  important  to  understand  associated  bioeffects. 
AFRL/RHDR  works  in  integrated  teams  including  researchers  from  the  Navy  and  Army, 
as  well  as  Joint  Non-Lethal  Weapons  Directorate  Programs.  This  tri-service  team 
provides  bioeffects  research  of  the  entire  electromagnetic  spectrum  covering  lasers  to 
radio  frequency  (RF).  Tri-service  DE  bioeffects  research  plays  an  important  role  in 
developing  world-wide  safety  standards  for  radio  frequency  and  lasers. 

Modeling  and  simulation  provides  a  powerful  tool  for  research  in  the  areas  of  DE 
bioeffects.  The  tri-services  team  jointly  developed  and  enhanced  finite  difference  time 
domain  (FDTD)  codes  for  solving  Maxwell’s  electromagnetic  equations  derived  from 
Eawrence  Eivermore  National  Tab’s  original  FDTD  code  [4]  and  the  implementation 
presented  by  Kunz  and  Euebbers  [5].  The  Brooks  FDTD  code  was  created  specifically  to 
calculate  the  specific  absorption  rate  of  biological  tissues  exposed  to  RF.  The  code  was 
validated  by  comparison  to  experimental  results  and  known  analytical  solutions  within 
geometrically  simple  models.  This  code  was  parallelized  at  Brooks  and  has  been  run  for 
over  four  years  on  a  Beowulf  cluster.  Modeling  realistic  anatomical  digital  models 
allowed  researchers  to  predict  areas  of  local  energy  deposition  within  the  human  body 
before  actual  experimentation,  thereby  improving  the  experimental  design  process. 
Modeling  allows  researchers  to  simulate  a  much  wider  range  of  frequencies,  pulses,  and 
sources  than  previously  attained  by  experimentation  alone  due  to  time  and  funding 
constraints. 


Background 

The  Finite  Difference  Time  Domain  (FDTD)  is  a  direct  time-domain  numerical  approach 
for  solving  Maxwell’s  equations  of  electrodynamics.  The  FDTD  method  was  first 
introduced  by  Kane  Yee  in  1966  [1],  and  has  become  increasingly  popular  in  the 
electromagnetic  modeling  and  simulation  community  as  available  computational 
resources  have  expanded. 

The  general  Yee  FDTD  algorithm  staggers  the  E  and  H-field  vector  components  within  a 
cubical  lattice.  The  grid  is  created  in  such  a  way  so  that  each  E-field  component  is 
surrounded  by  four  H-field  components  and  each  H-field  component  is  surrounded  by 
four  E-field  components,  as  shown  in  the  following  figure. 

The  Yee  algorithm  also  staggers  the  E  and  H-field  components  in  time,  so  that  they  are 
updated  in  a  leapfrog  manner.  The  current  value  of  an  E-field  at  a  point  is  calculated 
based  on  the  previous  value  of  the  E-field  at  that  point  and  from  the  previously  stored 
values  of  the  four  encircling  H-fields.  Next,  the  H-fields  are  updated  using  the  previous 
H-field  values  and  the  four  encircling  E-fields.  This  process  is  repeated  until  the  desired 
solution  state  has  been  reached. 
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Hv 


Figure  1 


FDTD  Update  Equations: 

Maxwell’s  equations  for  linear,  isotropie,  nondispersive,  lossy  materials  can  be  written 
as: 

—  =  --VxE--^  +(7*H 

"  source 
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=  +<7£' 
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These  equations  may  be  vectorized  and  written  in  terms  of  their  Cartesian  coordinate 
components.  For  example,  the  x-directed  component  equations  become: 
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Applying  Yee’s  finite  difference  scheme,  and  rearranging  terms,  the  FDTD  update 
equations  for  the  x-directed  E  and  H-field  components  may  be  written  as: 
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The  y  and  z-directed  update  equations  can  be  written  in  a  similar  manner. 


SAR  Calculations: 

The  Specific  Absorption  Rate  (SAR)  is  commonly  used  as  a  measure  of  the  energy 
absorption  within  biological  tissues.  The  SAR  can  be  calculated  by 

p 

where  p  is  the  density  in  Kg  per  cubic  meter,  a  is  the  conductivity,  and  E  is  the  root- 
mean-square  of  the  total  Electric  field. 

Eor  sinusoidal  steady-state  calculations,  the  EDTD  update  equations  are  typically  iterated 
until  the  SAR  has  reached  an  acceptable  convergence  criteria. 
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Boundary  Conditions 


Absorbing  Boundary  Conditions  (ABC)  are  often  used  with  the  Finite  Differenee  Time 
Domain  (FDTD)  method  in  order  to  simulate  the  extension  of  the  eomputational  zone  to 
infinity.  This  is  neeessary  in  order  to  properly  prediet  the  eleetromagnetie  fields  within 
an  unbounded  region.  In  the  past,  analytieal  teehniques  have  been  used  to  aehieve  the 
absorbing  boundary  eondition.  More  reeently,  the  Perfeetly  Matehed  Layer  (PML) 
boundary  eondition  has  been  introdueed  and  favored  in  many  FDTD  simulations.  The 
PML  is  an  absorbing  material  medium  that  ideally  will  mateh  plane  waves  of  any 
ineidenee  angle,  polarization,  or  frequeney. 

In  the  main  eomputational  spaee  of  an  FDTD  grid.  E-field  and  H-field  update  equations 
are  performed  in  a  leapfrog  manner  in  order  to  ealeulate  the  eleetromagnetie  fields  at  the 
eurrent  time  step.  Within  a  PML  boundary  layer,  in  addition  to  the  E-field  and  H-field 
updates,  the  eleetrie  flux  density  (D)  and  the  magnetie  flux  density  (B)  are  required.  The 
update  equations  for  the  x-direeted  eleetromagnetie  eomponents  within  the  PME 
boundary  are  given  below.  Similar  equations  apply  for  the  y-direeted  and  z-direeted 
eomponents. 
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A  polynomial  profile  is  applied  to  the  a  and  k  terms  within  the  PME  in  order  to  reduee 
refleetions  eaused  by  diseretization  errors.  The  value  of  these  eoeffieients  may  be 
ealeulated  at  a  given  depth  within  the  PML  based  on  the  following  equations. 
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In  equations  5  and  6,  d  is  the  depth  from  the  PML  surfaee.  The  polynomial  faetor  m  is 
typieally  given  a  value  between  3  and  4.  For  a  given  value  of  m,  ax,max  can  be  ealeulated 
by  the  following  equation,  whereas  Kx,max  is  often  just  set  to  1  within  the  PML. 


0.8(m  +  l) 


A  eomplete  derivation  and  deseription  of  the  implementation  of  a  PML  boundary  ean  be 
found  in  Taflove  pages  298-313  [6]. 


Results  and  Discussion 

Brooks  FDTD,  espeeially  the  Fortran  version,  has  been  validated  using  many  different 
methods  ineluding:  exaet  analytieal  solutions  of  simple  geometries,  eomparisons  against 
other  numerieal  methods  for  EM,  eomparisons  against  other  validated  FDTD  eodes,  and 
even  traditional  empirieal  experiments.  For  validation  purposes  of  this  teehnieal  report, 
we  eompare  both  Brooks  FDTD  implementations  (FORTRAN  and  C)  against  a  Mie 
theory  based  eode. 

In  our  test  setup,  we  eonsider  a  9em  sphere  made  of  a  tissue  equivalent  material  (TEM) 
being  exposed  to  a  plane  wave  ranging  in  frequeney  from  30  MHz  to  6  GHz.  In  our 
EDTD  runs,  we  will  use  both  1mm  and  0.5mm  resolution  spheres.  The  following  ehart 
depiets  the  EDTD  and  Mie  results  in  terms  of  whole  body  SAR: 
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The  chart  shows  a  tight  correlation  between  the  two  implementations  for  lower 
frequencies,  but  as  the  frequencies  increase,  FDTD  starts  to  diverge.  It  is  well  know  that 
the  FDTD  method  breaks  down  as  the  voxel  size  becomes  large  as  compared  to  the 
frequency  of  the  incident  field  [5].  Generally,  at  least  10  cells  per  wavelength  are 
recommended  for  accurate  results. 

Using  the  10  cells  per  wavelength  requirement,  the  1mm  sphere  model  and  the  0.5mm 
sphere  model  would  be  appropriate  for  frequencies  up  to  3.6  GHz  and  7.6  GHz, 
respectively.  In  this  case,  these  upper  bounds  have  errors  of  ~I0%  compared  to  the  MIE 
solution.  If  better  accuracy  is  required,  higher  resolution  models  would  be  advised. 


9 

Approved  for  public  release;  distribution  unlimited;  case  fide  no.  08-219,  10  Sep  08. 


Percent  Diffrence  From  Mie  (9cm  Sphere) 


CNj-^ini^ooococDcnojinoocMoi^Ln 


Frequency  (MHz) 


— .5mm  (Fortran  FDTD) 
1  mm  (Fortran  FDTD) 
^1mm(C  FDTD) 

.5mm  (C  FDTD) 


Comparing  the  two  implementations  of  the  Brooks  FDTD  code,  results  should  be  very 
close  as  the  algorithms  are  almost  identical.  Minor  deviations  in  their  results  would  be 
primarily  due  to  differences  in  compilers  and  math  libraries.  The  following  chart  plots 
the  percent  differences  between  the  results  of  the  C  and  Fortran  versions; 
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C  vs.  Fortran  FDTD  Percent  difference 


1mm 

.5mm 


Differences  between  the  C  and  Fortran  implementations  of  Brooks  FDTD  rarely  exceed 
1%  where  FDTD  is  known  to  be  stable. 

The  main  benefit  between  FDTD  and  exact  solutions  like  the  Mie  method  is  that  FDTD 
can  run  arbitrary  geometries.  The  following  are  SAR  results  from  the  Fortran  version  of 
the  Brooks  FDTD  code.  In  these  simulations,  RF  fields  with  frequencies  from  25  MHz  to 
3  GHz  were  incident  dorsally  upon  a  1mm  anatomical  man  model.  Vertical  electric  field 
polarization  was  assumed  for  the  incident  field.  The  graph  below  depicts  SAR  in  a 
handful  of  tissues  as  a  function  of  frequency. 
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1mm  Man  MEHK 


Frequency  (MHz) 

The  chart  shows  just  a  portion  of  the  tissues  found  in  the  1mm  anatomical  man  model. 
Brooks  FDTD  produces  a  full  SAR  output  for  every  1mm  voxel  of  the  model.  These 
results  can  be  visualized  by  any  number  of  tools. 

CONCLUSIONS 

The  Brooks  FDTD  FORTRAN  and  C  codes  were  observed  to  match  the  Mie  values  with 
a  very  small  margin  of  error  of  5%  for  frequencies  between  30  MHz  and  2.5  GHz  for 
1mm  resolution  models  and  between  30  MHz  and  4  GHz  for  0.5  mm  resolution  models. 
The  Brooks  FDTD  codes  had  a  margin  of  error  that  grew  as  the  frequencies  moved  above 
these  ranges  and  generally  could  not  produce  reliable  results  when  the  wavelength  is 
shorter  than  10  cells.  For  higher  frequencies  beyond  7.6  GHz,  FDTD  can  still  be 
employed,  but  models  at  a  higher  resolution  than  0.5  mm  are  required  for  accurate 
results. 

FDTD  has  been  proven  to  be  a  valuable  tool  for  modeling  RF  bioeffects.  With 
increasing  computing  capacity,  FDTD  can  be  taken  even  further  in  the  future  with  larger 
and  higher  resolution  models  of  arbitrary  geometries.  Modeling  with  FDTD  allows 
researchers  to  simulate  hundreds  or  even  thousands  of  scenarios  before  any  empirical 
experimentation  begins.  Brooks  FDTD  continues  to  be  developed  and  maintained  to 
ensure  accuracy  and  to  provide  useful  features  for  continued  RF  bioeffects  research. 
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Appendix  A:  Brooks  FDTD  Fortran  Source 

The  source  code  is  included  on  accompanying  CD  under  the  “src/fortran_version” 
directory. 
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Appendix  B:  Brooks  FDTD  (Fortran  Based)  User  Guide 

Install  Procedures 

Assumptions 

Compatible  Fortran  95  compiler 
Working  MPI  environment 

-  /usr/local  is  exported  to  all  nodes  in  the  cluster 

The  MPI  libraries  and  Fortran  library  paths  are  in  the  load  search  path  or  are  in  the 

“LD  LIBIRARY  PATH”  environment  variable 

Name  the  MPI  Fortran  90/95  wrapper  compiler  “mpif90” 

-  Log  in  as  root 

Building  Brooks  FDTD 

1 .  Uncompress  the  FDTD  source  files  into  a  temporary  directory 

2.  In  that  directory,  type  “make” 

3.  Copy  the  “fdtd”  binary  to  /usr/local/bin,  and  make  sure  permissions  are  set  to  555 
or  higher 

4.  Temporary  directory  and  files  may  be  deleted 


How  to  Run  FDTD  Simulations 
Introduction 

Assuming  you  have  configured  your  cluster  and  compiled  the  code  properly,  running  the 
code  is  relatively  simple.  Generally  most  problems  running  the  code  stem  from  cluster 
misconfigurations.  This  guide  will  step  you  through  running  sample  problems. 

Overview 

1 .  Command  Synopsis 

2.  Options  Description 

3.  Required  Files 

4.  RF  Source  Cardinal  Polarizations  for  plane  wave 

Command  Synopsis 

mpirun  [mpi  options  here]  fdtd  <  model.dat  -ifile 
antenna.dat  -t  tissue.txt 

[-air  n_layers]  [-r]  [-v  string]  [-eout]  [-hout]  [-t 

[gridlid]]  [-i  nbytes]  [-ps]  [-pml]  [?] 
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Options  Description 


9 

<  model.dat 

(required) 


-ifile  antenna.dat 

(required) 


-tfile  tissue.txt 

(required) 

-air  n_layers 


-r 

-V  string 
-eout 

-hout 

-t  [grid | id] 

-i  n_bytes 
-maxe 

-ps 


-pml 


Prints  usage 

Okay,  this  is  not  really  an  option,  but  redireetion  of  stdin; 
still,  it  is  neeessary  none  the  less,  “model.dat”  contains 
the  information  about  the  model  on  which  you  are  running 
the  simulation.  Refer  to  the  required  files  section  for  more 
information  as  to  what  goes  in  to  model.dat,  and  all  other 
required  files. 

Specifies  incident  field  file  or  antenna  file.  The 
antenna.dat  file  contains  information  describing  the  RF 
source.  For  this  version,  the  two  source  options  are  point 
source,  and  plane  wave. 

Specifies  the  tissue  file  that  relates  to  the  model  included. 

Adds  n  extra  layers  of  air  around  the  model.  For  example, 
if  n  equals  3  and  your  model  is  (21,  21,  21)  voxels^S,  then 
the  new  total  size  would  be  (27,  27,  27).  No  less  than  10 
layers  of  air  should  be  used.  If  the  -pml  option  is  not 
used,  additional  layers  may  be  required  depending  on  the 
frequency. 

Forces  root  to  calculate. 

Allows  “string”  to  be  appended  to  the  file  prefixes 
Outputs  the  E-field  values  in  x,  y,  and  z  at  the  end  of  the 
simulation. 

Outputs  the  H-field  values  in  x,  y,  and  z  at  the  end  of  the 
simulation. 

Unsuppressed  troubleshooting  output,  “grid”  will  print 
the  “gridMap”  assignments,  and  “id”  will  create  an  output 
file. 

Preallocates  n  bytes  of  memory  for  the  incident  field. 
Creates  an  output  file  that  contains  the  maximum  e  value 
for  each  cell  during  a  run 

Specifies  the  simulation  will  be  of  a  point  source,  and  not 
a  plane  wave.  The  point  source  configuration  file  must 
included  with  the  -ifile  option 

Specifies  that  an  eight  layer  PML  absorbing  boundary 
condition  (ABC)  is  to  be  used  instead  of  the  regular  one 
layer  ABC. 


Required  Files 

The  ‘#’  characters  and  following  text  are  included  in  the  examples  only  for  descriptive 
purposes,  and  should  not  be  included  in  you  actual  files.  Not  all  routines  were  written  to 
handle  comments,  and  some  fields  are  hard  coded  to  read  in  a  fixed  amount  number  of 
characters. 
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model.dat 

Describes  the  model  and  the  frequencies  to  be  run. 


tissue . txt 

#Name  of  tissue  properties  file  to  be  used 

./ 

#Output  Directory 

Dirpre 

Test 

#Output  file  prefix 

.  This  will 

be  prepended  to 

all  output 

.05 

#Cutoff  for  ratio  diff  from  1. 

0 

crit 

4 

#Minimum  number  of  ] 

half  cycles 

to  run 

minhcyc 

10 

#Min  Time  steps  per 

wavelength 

dtperwave 

Testrawf ile , 

.raw  #Model  file 

rawf ile 

10 

#The  X  Dimension  of 

the  Model 

Orientation 

nx 

10 

#The  Y  Dimension  of 

the  Model 

Orientation 

ny 

10 

#The  Z  Dimension  of 

the  Model 

Orientation 

nz 

0.002 

#Voxel  size  in  X  in 

meters 

0.002 

#Voxel  size  in  Y  in 

meters 

0.002 

#Voxel  size  in  Z  in 

meters 

2060  . 

#Frequency  in  MHz 

3000  . 

#Multiple  frequencies  can  be  listed  here 

used.  The  following  types  of  antenna/sources  are  available  with  this  version  of  FDTD; 
point  source  and  plane  wave. 


antenna.dat  for  plane  wave 

The  vector  to  the  plane  wave  source  is  found  using  the  theta  and  phi  method. 

Imagine  yourself  in  the  x-z  plane  with  your  feet  in  the  +z  direction,  and  your  left  side 
in  the  +x  direction.  Take  your  right  arm  and  rotate  it  from  -z  to  +z  in  the  x-z  plane 
for  theta.  Next,  rotate  your  arm  counter-clockwise  for  the  phi.  For  example  if  your 
vector  is  theta=90,  and  phi=90,  the  vector  would  first  rotate  from  -z  to  -x  for 
theta=90,  then  it  would  rotate  from  -x  to  +y  for  phi.  The  result  is  a  vector  going  in 
the  +y  direction. _ 


90.0 

#Theta  latitude  (degrees) 

thine 

0.0 

#Phi  longitude  (degrees) 

phinc 

1.0 

#E  theta 

ethinc 

0.0 

#E  phi 

epthinc 

1.0 

#Magnitude 

antenna.dat  for  point 

source  (sinusoidal) 

50 

#x  location  of 

ps 

100 

#y  location  of 

ps 

500 

#z  location  of 

ps 

1.0 

#Amplitude 

1 

#Orientation 

0 

#ps  type  (0  == 

sinusoidal,  1  ==  Gaussian  pulse) 
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antenna.dat  for  point  source  (Gaussian  pulse) 


50 

#x  location  of 

ps 

100 

#y  location  of 

ps 

500 

#z  location  of 

ps 

1.0 

#Amplitude 

1 

#Orientation 

1 

#ps  type  (0  == 

sinusoidal,  1  ==  Gaussian  pulse) 

200.0 

#n  sub  0 

50.0 

#n  decay 

Detines  the  dielectric  properties  ot  the  tissue  types  tound  in  the  raw  tile. 


AIR.  (external) .  9  24  135  .0013  l.OE+00  O.OE+00  l.OE+00  1.00 

O.OE+00  l.OE+00  1.00  O.OE+00  l.OE+00  1.00  O.OE+00  l.OE+00  1.00  0.0000 
0 

AIR.  (internal) .  0  0  0  .00130  l.OE+00  O.OE+00  l.OE+00  1.00  O.OE+00 

l.OE+00  1.00  O.OE+00  l.OE+00  1.00  O.OE+00  l.OE+00  1.00  0.0000  1 

BILE .  105  25  132  1.01000  4.0E+00  6.6E+01  2.1E+10  .95 

5.0E+01  l.OE+08  1.00  O.OE+00  l.OE+03  .80  O.OE+00  l.OE+01  .80  1.4000  2 

BLADDER .  0  128  0  1.03000  2.5E+00  1.6E+01  1.8E+10  .90 

4.0E+02  l.OE+06  .90  l.OE+05  l.OE+03  .80  l.OE+07  l.OE+01  1.00  .2000  227 

BLOOD .  0  255  146  1.05800  4.0E+00  5.6E+01  1.9E+10  .90 

5.2E+03  1.2E+06  .90  O.OE+00  l.OE+03  .80  O.OE+00  l.OE+01  1.00  .7000  189 

BLOOD. VESSEL .  200  100  200  1.04000  4.0E+00  4.0E+01  1.8E+10  .90 

5.0E+01  5.0E+07  .90  l.OE+05  l.OE+03  .80  l.OE+07  l.OE+02  1.00  .2500  65 

# - 1 - I  _2-  I  -3-  I  -4-  I  ---5---  I  ---6---  I I  ---8---  |  -9-  1 

--10 - I  --11 - I  -12  i  --13 - I  --14 - 1-15  1  --16 - 1  --17 I  -18-  I  --19-  i  20 

#more  tissues  could  be  listed  here 


1. 

2.  Label 

3.  R  of  the  RGB  value 

4.  G  of  the  RGB  value 

5.  B  of  the  RGB  value 

6.  Density  (only  used  for  SAR  calculations) 

7.  Epsilon  infinity 

8.  dell 

9.  l/(2pi  *  taul) 

10.  (1-alfl) 

11.  del2 

12.  1/(2  pi  *  tau2) 

13.  (l-alf2) 

14.  del3 

15.  l/(2pi  *  tau3) 

16.  (l-alf3) 

17.  del4 

18.  l/(2pi  *  tau4) 

19.  (l-alf4) 

20.  Oi 

21.  ID  Number 
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Where,  So  =  8.854*10^-12  and  ro  =  2*pi*freq, 

Relative  Permittivity  =  Re  a  /  [J  Qo 
Conductivity  =  2*  tt*  eps Q*freq*  imaginary  \J} 

and, 


del , 


del-, 


del- 


del- 


XU^cox  inq^cor^D'^^  ID^'cot^D'^' 


RF  Source  Cardinal  Polarizations  for  Plane  Wave 


EDTD  Parameters 

Anatomical  Orientation 

Polarization 

Theta 

Phi 

ThetaPolar 

PhiPolar 

Flead/ 

Right 

Eront 

Tail 

/  Eeft 

/  Back 

PEHK 

90 

90 

1 

0 

E 

H 

E>B 

MEHK 

90 

270 

1 

0 

E 

H 

B>E 

PEKH 

90 

0 

1 

0 

E 

E>R 

H 

MEKH 

90 

180 

1 

0 

E 

R>L 

H 

PHEK 

90 

90 

0 

1 

H 

E 

E>B 

MHEK 

90 

270 

0 

1 

H 

E 

B>E 

PHKE 

90 

0 

0 

1 

H 

E>R 

E 

MHKE 

90 

180 

0 

1 

H 

R>E 

E 

PKEH 

0 

0 

1 

0 

H>T 

E 

H 

MKEH 

180 

0 

1 

0 

T>H 

E 

H 

PKHE 

0 

0 

0 

1 

H>T 

H 

E 

MKHE 

180 

0 

0 

1 

T>H 

H 

E 

K  is  listed  by  H>T,  T>H,  L>R,  R>L,  F>B,  or  B>F  where  H  means  head,  T  tail,  L  left,  R 
Right,  F  front,  and  B  back.  These  are  meant  to  signify  the  direction  of  K;  for  example, 
H>T  means  head  to  tail.  Front  refers  to  the  human  front,  or  anatomically  the  ventral  side. 
There  is  not  a  single  unique  combination  of  parameters  for  each  orientation,  and  the 
program  will  accept  other  parameters  including  values  between  0.0,  and  1.0  for 
ThetaPolar,  and  PhiPolar,  but  all  values  have  not  been  tested,  so  be  careful. 

Assumptions:  x  is  side  to  side,  y  is  front  to  back,  and  z  is  head  to  tail. 
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Appendix  C:  Brooks  FDTD  C  Source 

The  source  code  is  included  on  accompanying  CD  under  the  “src/c_version”  directory. 
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Appendix  D:  Brooks  FDTD  (C  Based)  User  Guide 

Installing  FDTD: 

Compiling  FDTD  should  be  a  simple  affair  assuming  a  C  compiler,  MPI  libraries,  and  a 
properly  configured  environment.  This  FDTD  implementation  has  been  compiled 
without  problems  with  GCC  gee  4*,  Intel  ice  8-10,  and  PGI  pgcc  6-7  with  MPI 
implementations  from  LAM-MPI,  OpenMPI,  and  Cray. 

To  compile  FDTD  with  the  Makefile,  type  in  the  directory  of  the  source  code: 

$  make 

To  compile  FDTD  without  the  Makefile,  type: 

$  mpicc  -o  fdtd  *.c  -Iz  -Im  -Irt  -03 

To  install  FDTD,  simply  copy  the  binary  to  anywhere  that  is  in  your  path  and  is 
accessible  to  all  the  nodes  on  your  cluster.  For  example,  if  /usr/local  is  exported  to  all 
nodes,  you  could  copy  the  binary  to  /usr/local/bin,  or  if  you  don’t  have  access  to  copy  it 
there,  you  can  copy  it  to  your  own  personal  -/bin  directory. 

#  cp  ./fdtd  /usr/local/bin 

-or- 

$  cp  ./fdtd  ~/bin 

Input: 

Most  of  FDTD’ s  features  are  controllable  through  command  line  options. 

Command  Line  Options: 

Options  Always  Required: 

•  — raw/-r  [raw  file  name] 

o  The  main  model  file  that  simulation  will  run  against 

•  —freq/-f  [frequency] 

o  Used  to  calculate  the  dielectric  properties  of  the  tissue  types 
o  Used  for  the  source  if  it  requires  a  frequency 

•  — tissue/-t  [tissue  file  name] 

o  Tissue/material  properties  file 
Options  Required  for  a  Plane  Wave  Incident  Field: 

•  — pw/-P  [theta,phi,theta_amp,phi_amp] 

o  A  file  name  with  the  same  parameters  one  per  line  is  also  acceptable 
Additional  Options 

•  — air/-a  [number  of  air  layers] 
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o  If  unspecified,  10  layers  of  air  are  added  on  by  default 
o  It  is  not  recommended  to  have  less  than  1  layer  of  air 

•  — delta/-d  [dx,dy,dz] 

o  specifies  the  cell  size  in  meters 
o  Default  cell  size  values  are  0.001,0.001,0.001  m 

•  — maxe/-m 

o  Tracks  the  maximum  E-field  values  and  produces  a  SAR  like  file  for  all 
cells  in  the  model 

•  — pml/-p  [number  of  PML  layers] 

o  Default  number  of  PML  layers  is  8 

o  Less  than  4  layers  is  not  recommended,  and  less  than  1  will  not  work. 

•  —prefix/-x  [output  prefix] 

o  Provides  the  prefix  for  the  output  files 

o  The  default  prefix  is  the  raw  file  name  minus  the  “.raw”  ending 

•  — iterations/-i  [max  number  of  iterations] 

o  The  maximum  number  of  iterations  or  time  steps  to  run 

•  — logfileZ-l  [log  file  name] 

o  Defaults  to  [prefix]. log 

•  —  verboseZ-v 

o  More  output  to  the  shell 

•  — helpZ-hZ-? 

o  Prints  out  usage 

RAW  File: 

The  RAW  file  contains  the  model  or  geometry.  It  is  3D  volumetric  data  stored  as  a  ID 
binary  stream  of  bytesZchars.  Each  byte  cast  to  an  unsigned  integer  corresponds  to  one  of 
the  possible  256  tissue  type  IDs  found  in  the  tissue  file.  The  ID  stream  is  organized  in 
increasing  order  first  in  the  x,  then  y,  and  finally  z  directions.  Liles  of  this  format  can  be 
read  and  written  with  the  following  algorithm: 

for(k  =0;  k  <  dim_z;  k++) 

for(j  =0;  j  <  dim_y;  j+  +  ) 

for(i  =0;  i  <  dim_x;  i++) 

data[i] [j] [k]  =  read (file, sizeof (type) ) ; 

LDTD  expects  the  filename  to  contain  the  dimensions  of  the  file  in  the  following  format: 

x[interger]y[integer]z[integer].raw.  Lor  example,  a  file  named 

“sphere  xOlOlyOlOlzOlOl.raw”  would  specify  the  mode  to  be  101x101x101  cells. 
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Tissue  File: 


The  tissue  file  contains  the  dielectric  properties  of  all  of  the  materials  found  in  the  raw 
file.  The  file  is  organized  with  20  comma  separated  columns  and  a  row  per  tissue  type. 
The  columns  in  order  are: 


1 . 

Material  name 

2  . 

R  (red) 

3. 

G  (green) 

4  . 

B  (blue) 

5. 

specific  density 

6. 

or  infinite  dielectric  value 

7  . 

Asi 

8  . 

fi  or  dispersion 

frequency 

1 

9. 

o 
\ — 1 

As2 

11 . 

f2  or  dispersion 

frequency 

2 

12  . 

Oi2 

13. 

As3 

14  . 

fa  or  dispersion 

frequency 

3 

15. 

aa 

16. 

As4 

17  . 

f4  or  dispersion 

frequency 

4 

18  . 

04 

19. 

Qo  or  DC  conductivity 

20.  Tissue  ID 


Output 

FDTD  outputs  several  files:  specific  absorption  rate  (SAR),  log  file,  and  maximum  E- 
field  file  if  the  — maxe  options  was  used.  The  sar  and  maxe  3D  volumetric  data  as  a  ID 
binary  stream  of  floats  organized  in  the  same  way  as  the  raw  file. 

Running  FDTD 

FDTD  was  designed  to  be  executed  in  parallel  on  a  Beowulf  type  cluster,  but  anything 
with  a  C  compiler  and  a  MPI  implementation  should  work. 

Basic  execution  of  FDTD: 

mpirun  [mpi  options]  fdtd  [fdtd  options] 

The  following  is  an  example  of  how  to  run  a  plane  wave  on  a  sphere  on  2  processors: 
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mpirun  -np  2  fdtd  -r  Sphere_x002 ly002 1 z002 1 . raw  -t 
tissue.txt  -f  3000  -P  90,0,1,0  -v 

Example  Torque  script  to  submit  a  FDTD  run  to  a  Torque/PBS  based  resource 
management  system: 

# ! /bin/bash 

#PBS  -1  nodes=2 : ppn=2 

mpirun  fdtd  -t  tissue.txt  -r  sphere_x0120y0120z0120 . raw  -v 
-f  500  --pw  90,0,1,0  -1  test_log.out  -a  10  --prefix  job_8 
exit  0 
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Appendix  E:  Model  Files 


The  model  files  are  included  on  accompanying  CD  under  the  “models”  directory.  Model 
files  are  stored  in  the  RAW  format  as  discussed  in  Appendix  D  under  the  heading  “RAW 
Files.”  Generally  the  file  name  describes  what  the  RAW  file  is;  for  example, 
man_lmm_x0586y0340z  1878. raw  specifies  the  visible  man  at  a  1mm  resolution. 
Typically  if  no  resolution  is  specified,  1mm  is  assumed.  The  corresponding  dielectric 
properties  for  each  model  voxel  is  located  in  “/models/tissue. txt”. 
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